Assigning Grade Levels to Textbooks: Is it just Readability?

We evaluated the effectiveness of new indices of text cohesion to determine the appropriate human assigned grade level of a text. In particular, we investigated the efficacy of automated text indices produced by the online tool CohMetrix in predicting the grade level assigned by publishers to their own textbooks. To do this, we sampled 311 school textbooks from a large database, choosing roughly equal numbers of science, narrative, and social science texts. Publisher-assigned grade levels were found to be moderately predictable by traditional approaches such as the FleschKincaid Grade Level. Prediction of grade level was significantly improved by the inclusion of cohesion indices obtained by Coh-Metrix. Implications for the improvement of textbook selection are discussed.